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ABSTBACT 

Becent events in the world of evaluation suggest the 
need to reassess the basic purposes that justify its existence* In 
the long run, the language of evaluation eust be shaped by the nature 
and intents of the audience and by the nature of vhat is being 
evaluated. It is the nature of vhab is being evaluated that aust 
detereine the aethodologiesv processesf and reporting foraats of 
evaluation. To too aany people, evaluation still aeans seasureaent. 
According to one authorityv aeasureaent basically involves the use of 
nuaerical values to represent attributes of objects; assessaent 
includes aeasureaent plus those judgeaental activities that deter sine 
vhat and how to aeasures and soae aspects of evaluation are outside 
the reala of both aeasureaent and assessaent. The nature of vhat is 
being exaained and the purpose of the exaaination deteralne whether 
one is involved in aeasureaentv assessaent » or evaluation. The 
dissatisfaction with defining every evaluation problea as a 
aeasureaent problea is plain. The recognition that the nature of the 
problea has an inportant bearittg on the choice of aethod is 
encouraging. (Author/J6) 
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THE FRACTZCI Of EVAUMLXZOH AS A 

H. Richard SMCk 
Of fle« of lattruetieiMl BMoureM 
Uiiiv«rsity of XXUnolt UrtaBa^-Chavaisn 

Ifeimtains of wovdf htvm koon wrltttn tbovt nhtt tivAliiatioii Is and 
bow ovaluatort aighfi precood with Choir work. EvoluatioB Mdolf oro as 
prolific aa Mkaa and aiiaa of autowbilaa - and both prodiicta ara 
toutad in about tha aaat way* Cartainly tba idaa of a yaarly chaagaovar 
aaaaa to bava baeoaa findy antranchad in tba fiald of avaluatiout and 
for avaluatora tba annual stating of tba iteirieaa fidueational Kaaaarab 
Aaaoeiation baa baeoaa tba tquivaUnt of tba Saw Torfc Xntatnatioaal Auto* 
■Dtiva Bifalbition. And lika tba aatoaobilaa tpawad forth by tba graat 
Datroit aaabinaa. avaluatioa aodala bava troim prograaaivaly graadar* 
prograaaivaly aora conplax, and prograaaivaly nora aapaaaiva to oparata. 
In an affluent ooeiaty of acaaiagly unliaitad rasourcaat larga. coaplaK» 
and axpanaiva autoonbilaa ataatd to auit our atyla of living. 2h 
tba oaaa way, in tba daya of aamlngly andlasa govamaant affluonea 
ancouraging tba aaployaent of avaluatora* larga» eoaplas, and aapanaiva 
avaluationa oaoaa to bava auitad tba atyla of living to idiieb ua uould 
all lika to ba aeeuatoatd. But raeant avanto in both tba iiorld of tba 
tranaportation industry and in tha world of avaluation auggaat that tba 
tiae baa cons for aach to rsaassss tha basic purpoaas on which ita 
axiatanca ia Juatifiad. Both bava baan feading off thair aarkating 
auceassas for so long tbst tha sssuaptions on which thair axiatanca 
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dtpMdt havtt tetn largftly lorfoteta. A rMlltttlon of thm finite 
Mttirt of our uttnral rtooureM will etrtaialy eatwo tho traaoporta- 
cioa induftry to faea a largar raality than chat proviM by tha pr^if it 
ttotlva, and aa inpatiaiit ptiblle, quaatloning tha valua of amy aeadaaie 
purauita, teiaga ua faea to faca with iaauaa about tha aada rathar 
than tha aaaaa of «ur Avaluativa afforta* 

Faw of ua would classify ouraalvaa as oparating in cha world 
of "basic" aclsnea. Hoat avaluators arc quits satiafiad with tha notion 
of thsir vorfc aa an appliad f iald sarvlng sn and rslatad to tha aduca* 
tional andaavor. Evaluation has purposa • and a practical ona at that* ' 
A problan of baconing throughly angroasad in any f iald of atudy ia 
that ona is lad inavitably to a praoeeupation with aaana and, aa long 
as othara ara willing to aupport us» wa can lusuriata in tha intallactual 
atisulation of avar nora cogently defining and refining our procadurea* 
In the proceas our language becones nore and a»re esoteric. In the 
process the elegence of our nethodology takes prec e dence over the pur- 
poses of our aetivitiea. Our preoccupation with asana haa coat ua 
dearly. A United Stetea Senator recently told e group of educational 
researChera it waa uareaaonable to aasuaa thet there ia support for 
education reaearch and davelopnent in our country. The Senator apelled 
out one reaaon for a lack of political support. 

Educatora often speak a language of their own, one which 
ia unintelligible to the uninitiated. It ia very unrealistic 
for the education eatablishasnt to cone to the Hillf speak ar- 
cane words and phrasea» and then react in an offended oaonar when 
they are not underatood. When teatiaony ia given* it wist be 
in siaple leaguege that the generelist esn conpr^end» or other- 
wise your story will never get acroaa (Pell, 1975). 

tfould anyone deny that the lenguaga of evaluation ia any leaa 
arcane than that of other educational apecialisations? But if we ere 
being less than successful in eooBunicating with our benefactors are 
we doing batter with our clients? It was Stake (1972a} who said. 
■saaureaeata are pralaed often by iqr colleaguea, seldon by mr clianU.^' 
I don't think nuch haa changed since then* and part of the ptobX-^ 
relates to the language avaluators use to coionunicate with their au- 
diencea. Our choice of language in part relates to our lack of agreeaent 
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• ' . *• 

About tiM maim to b« aehltved by •valuation. If it la poaalbla to agraa 
about tha anda^j^o ba aarvad through an activity* tfia naeaaaary aaana 
of attaliiliit4thoaa aidia will baeoaa aora daar. Z ballava that la tha 
long run tha laii8uaga*«(|^ayaluatloii.i&ust ba ahjipad by (1) tha natura 
and Intanta of tha audlaneaa eonaldavlng avaluatlsa Infotoatlon and 
(2) tha nature of idiat la being evaluated. 

The Nature and Intanta of tha Audlancaa Cenalderlng Bveluatlva Inforaatlon 

Stake (1972b) haa throu^ nuah of hla writing aenaltlaed eval* 
uatora for tha need to conaldar the eudleneea addreaaed by an evaluation. 
Scrlven (1967) haa dlacuaeed at length the rolea and goala of avaluatloat 
and partially relatea the rolea of evaluation to the purpoaea of the 
audience. Perhapa Stuff ICbaam (1971) haa made us aoat keenly aware of tha 
need for quality evaluatlona to have credibility, tinallneaa and 
laportance. It waa also Stuff labean idio offered a definition of eval- 
uation as being the process of delineating, obtaining, end providing 
useful Infonatlon for Judging "decision altematlvea.*' So we have 
had over thl'^easa ^authors reninding us to consider our audiences, to 
consider the roles as well aa the goals that evaluation plays, to 
attend to attributes that laake evaluatlona uaeful to eudleneea, and 
that the proper use of evaluation is as a tool in the declsion-msklng 
procsfis. All of these ere inportant points. None, except Stufflebeaa'a 
auggetitlon that eveluatlona are for deciaion-naklng purposes suffi- 
ciently address the question of the uses to .which audiences actually 
put eveluatlon Infomation. Sevexal of the purposes for which I 
have seen audiences use evaluations prove considerably aore cos^lex 
and also considerably aore interesting than sinply using evaluation aa 
a tool for declnion making. Allow me to list a few: 

1. Ev<>litf . ttion as problem solver . As one source of legitimate 
information an e<^aatlon often serves as a stepping stone to problem 
solving. Formative evaluations can be extremely influential la 
developmental activities* And summatlve evaluation has proven helpful 
in estimating the quality t^nd pinpointing problems in a variety of 
progrems. Good evaluations that are atteniSive to the iiser's need are 
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1i«lpful ia aUtvittlng prebltM. Uaiag •valtittlMi rtpprt* to litlp 
allsvlAte probl«M it qultt dif f trent from uiing th«i m dteiiioii 
MkAM. Zt*9 for dl«logiM» not doeisiont, that ovaluationa find 
tboir Boot eoaitnwtlva uao, 

2. Bvlnation %o chanao f^nt. Both tha proeaaa and tha product 
of avaluatlon of taa caa act aa a ehanga agaat in particular aituationa. 
Fomaily docuaaatiiig and deacribing what ia alraady part of tha iaforaal 
eonauoieatioa natwork caa hava a powarful iapatua for chaaga. Zf tha 
avaluation ia dona by aa "outaidar" rathar than aa "iaaidar" thia 
docuaantatioa caa hava aa avan aora profound af fact. 

3* Evaluation aa tha davil'e advocata. Ona nay that avaluationa 
act aa changa aganta ia by allowing thanalavaa to ba uaad aa "tha 
davil'a advocata." In aa obvioualy uarafaraaead lattar froa aa 
avaluator to a eliant» tha following aantaaca appaarad. **Your rola 
would ba that of a racipiant of an avaluator *b racoaaaadatioa, rathar 
than that of diiaf architect of a davil'a plan.'* Balag abla to 
poiat to aa avaluatioa raport aa tha iaatigator for diaaga ia a nora 
coBBoa uaa of avaluatioa thaa oaay avaluatora raaliaa. fBioavar ia 
advocatiag tha dhanga can ba laaa paraosaUy and aubjaetivaly involvad. 

4. gyaluation aa oonvantion . Evary projact naada ita evaluation. 
And avaryona knowa that avaluatiooBmat ba dona in a car tain way 

if thay ara to ba raapactabla. Bouaa (1973) haa apallad oat in aoaa 
detail the conatraining forcaa of a technology of evaluation rooted in 
aaaaureaent and the conventional aature of evaluationo eaanatiag froa 
that technology. He htm reainded ua that govemaental ageaciea only 
accept "hard" data. Yet few ere cosq^llaenting evaluatora for their 
cogent end uaaful evaluationa. Uaad aa convention* evaluation reporta 
becoae bulk aaterial for our filea. 

5. Evaluation aa liturev . Mian uaed in thia way it doean't 
natter what ia aaid in the evaluation report, and the aore ercene tne 
language the better. It 'a enough to be able to aey that an avaluatioa 
waa done* that thia or that eninent evaluation aenter waa involved, 
end thet the reeulta ara on file. Ife have oede our bow to the elter 
of the evaluation goda. The liturgical uaa of evaluationa is probably 
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v«Ue«d to th« utility valut of p—t tvaluatient •spAviaaead by 
the aiklimet, or to tho dostoo to irtiich thoy bolitvo thoy had bottor 
rovoco *'ovtltt«tioa." 

Thoro la a hairarcfay auggaatad by tba abeva uaaa to iihieb 
elianta put avaluationa. Tha lovar on that hairardiy our avalua* 
tiona fall, tha laaa likaly it ia that tha flald of avaluatioa la 
aarvlng a purpoaa Which will aaaura ita aurvival* 

Tha Natura of tJhat la Baing Evaluatad 

A qualitative avaluatlon of acadanic dapartnanta ia quita 
dif faraat from evaluating tha af faetivanaaa of aupplanantary vatarial 
developed for a particular eouraa. Searching for tha atraagtha and 
neaknaasea in a curriculun ia of a different order than helping 
an Inatructor evaluate hia or her teaching with an aye toward inprova^ 
nant. The coaonn eleaent in each of tha above probleaa ia alaply that 
each can be evaluated. It is the nature of what ia being evaluated 
which Buat detemine aethodologiea, proceaaea, and reporting formate 
of evaluation. To too many people evaluation atill equala maaaura- 
ment. In spite of Scriven'a (1967) dear atatsoent that evaluation 
involvea the estimation of merit, of worth, or of value, and in apita 
of ftake'a (1969) enconpaaaing statement that, "All evaluation deala 
aicplicitly with the worth of aonathing," we have not been able to 
ahake the belief that evaluation and meaaurement are aynonomoua terms. 

Tli»rdrop (1972) haa delineated the concept of meaaurement from 
that of avsessnent on one hand and from evaluation on the other. By 
his reasoning, messurement basically involves the use of npimerical 
values to represent attributea of objecta. An attribute^ in order to 
be measurable, must fit the apeeificatlona of a quantitative variabla, 
and aome unit of aeaoureneat must be eatabliahed. Aaaeaament includaa 
measurement, but additionally involvea thoae qualitative and judgemental 
activitiea which go into determining what and how to measure. Bis 
reasoning concludes that there are some aapecta of evaluation which 
are outaide the realm of both meaaurement and assessment. Tha 
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Mittar* of ifhat it btint •smiMAt tsd cht pucpost pf tlm tMaiMtioAt 
ttft th« 4«t«niiiitr8 of vtethtr eat it itvolvtd in t nttturtBtttt 
tttttamt&t or tvtluatloii aetlvlty* Utrdrop't dittinotiont trt vorthy 
oatt* in teetpttaet of hit dittlncti^nt could furthtr our tffortt to 

■ 

dtvtlop t Itaguagt for tvtluttion rtUttd aert diroctly to itt purpottt. 

Thtrt dett imb to bt « gvoifliig rteognition that eoattatrt ara 
aora and aora doaoding raaulta from avaluation that hava a raal utility 
valaa rathar than raaulta that oaraly raf laet tha baat afforta of a 
aaaauramant tachnology. Flaughar (1974) ia oaa of tha aora raoant coa^ 
varta to a laat rigid viav of avaluatioas 

** • • • aducatiooal avaluation proparly dona ia a 
congloaaration of approxiaationa to tha idaal» abtruaiva 
attampta to ba unobtrutiva, Xtatueky «indaga» guaatvorh 
and vithful thinking. • • • 

"Tha aoat affaetiva» utaful avaluation ia aerappy» 
rough» patefavorfct fraquantly incoaplata and oripplad by 
unraaliatic tlaa daadllaaa whioh raquira ahort tarn 
guaaaaa about raaehing long tarn goAla. It ia dif ftrtat 
for avary nev aedia intreducadi eurricttluB» taxtt fila» 
coaputar attittad inttruetion." 

Straaga vorda indaad f roa a aanior raaaareb ptychologiat with Educational 
Tooting Sarvica. 8tranga—but encouraging. Bouaa (1979) haa goaa ao far 
aa to auggaat thara ia a "couatareultura" ia avaluation which haa avolvad 
in diract raaponaa to apaeifie aseaaaaa and daf icianciaa of aodam avalua* 
tion tachaology. And Provua (1973) haa gona ona atap furthar in auggaat- 
ing that, "A now profaaaional ia naadad. A nav tyataa ia naadad ahich 
coabinaa avaluation* conaun^Lty davalopaent» avaluation aathodology» and 
public inforattion into a wiifiad ahola." 

Tha diaoatiafactlon with defining avary avaluation problaa aa a 
aaaaureaaat problaa la plain. Tha racognition that tha nature of tha 
problaa haa an laportant bearing on tha choice of aathod io an 
encouraging one. 
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